GEMINI: A Generic Multi-Modal Natural Interface Framework for Videogames

In recent years videogame companies have recognized the role of player engagement as a major factor in user experience and enjoyment. This encouraged a greater investment in new types of game controllers such as the WiiMote, Rock Band instruments and the Kinect. However, the native software of these controllers was not originally designed to be used in other game applications. This work addresses this issue by building a middleware framework, which maps body poses or voice commands to actions in any game. This not only warrants a more natural and customized user-experience but it also defines an interoperable virtual controller. In this version of the framework, body poses and voice commands are respectively recognized through the Kinect's built-in cameras and microphones. The acquired data is then translated into the native interaction scheme in real time using a lightweight method based on spatial restrictions. The system is also prepared to use Nintendo's Wiimote as an auxiliary and unobtrusive gamepad for physically or verbally impractical commands. System validation was performed by analyzing the performance of certain tasks and examining user reports. Both confirmed this approach as a practical and alluring alternative to the game's native interaction scheme. In sum, this framework provides a game-controlling tool that is totally customizable and very flexible, thus expanding the market of game consumers.Comment: WorldCIST'13 Internacional Conferenc

Use of Markov processes in writing recognition

In this paper, we present a brief survey on the use of different types of Markov models in writing recognition . Recognition is done by a posteriori pattern class probability calculus . This computation implies several terms which, according to the dependency hypotheses akin to the considered application, can be decomposed in elementary conditional probabilities . Under the assumption that the pattern may be modeled as a uni- or two-dimensional stochastic process (random field) presenting Markovian properties, local maximisations of these probabilities result in maximum pattern likelihood . We have studied throughout the article several cases of subpattern probability conditioning. Each case is accompanied by practical illustrations related to the field of writing recognition .Dans cet article, nous présentons une étude sur l'emploi de différents types de modèles de Markov en reconnaissance de l'écriture. La reconnaissance est obtenue par calcul de la probabilité a posteriori de la classe d'une forme. Ce calcul fait intervenir plusieurs termes qui, suivant certaines hypothèses de dépendance liées à l'application traitée, peuvent se décomposer en probabilités conditionnelles élémentaires. Si l'on suppose que la forme suit un processus stochastique uni- ou bidimensionnel qui de plus vérifie les propriétés de Markov, alors la maximisation locale de ces probabilités permet l'atteinte d'un maximum de la vraisemblance de la forme. Nous avons étudié plusieurs cas de conditionnement des probabilités élémentaires des sous-formes. Chaque étude est accompagnée d'illustrations pratiques relatives au domaine de la reconnaissance de l'écriture imprimée et/ou manuscrite

Deep Learning for Single-Molecule Science

Exploring and making predictions based on single-molecule data can be challenging, not only due to the sheer size of the datasets, but also because a priori knowledge about the signal characteristics is typically limited and poor signal-to-noise ratio. For example, hypothesis-driven data exploration, informed by an expectation of the signal characteristics, can lead to interpretation bias or loss of information. Equally, even when the different data categories are known, e.g., the four bases in DNA sequencing, it is often difficult to know how to make best use of the available information content. The latest developments in Machine Learning (ML), so-called Deep Learning (DL) offers an interesting, new avenues to address such challenges. In some applications, such as speech and image recognition, DL has been able to outperform conventional Machine Learning strategies and even human performance. However, to date DL has not been applied much in single-molecule science, presumably in part because relatively little is known about the 'internal workings' of such DL tools within single-molecule science as a field. In this Tutorial, we make an attempt to illustrate in a step-by-step guide how one of those, a Convolutional Neural Network, may be used for base calling in DNA sequencing applications. We compare it with a Support Vector Machine as a more conventional ML method, and and discuss some of the strengths and weaknesses of the approach. In particular, a 'deep' neural network has many features of a 'black box', which has important implications on how we look at and interpret data

Critical weather limits for paddy rice under diverse ecosystems of India

Rice yields are largely influenced by variability in weather. Here, we demonstrate the effect of weather variables viz., maximum and minimum temperatures, rainfall, morning and evening relative humidity, bright sunshine hours on the yield of rice cv. Swarna, grown across five rice ecologies of India through field experiments during kharif (wet) season (Jun-Sept.). Critical thresholds of weather elements were identified for achieving above average, average and below average yield for each ecology. The investigation could determine how different weather elements individually and collectively affect rice yield in different rice ecosystems of India. While a sudden increase in minimum temperature by 8-10 °C (> 30 °C) during reproductive period resulted in 40-50 per cent yield reduction at Mohanpur, a sudden decrease (< 20 °C) caused yield decline at Dapoli. The higher yields may be attributed to a significant difference in bright sunshine hours between reproductive phases of above-average and below-average yield years (ranging from 2.8 to 7.8 hours during P5 stages and 1.7 to 5.1 during P4 stages). Rice cultivar Swarna performed differently at various sowing dates in a location as well as across locations (6650 kg ha-1 at Dapoli to 1101 kg ha-1 at Samastipur). It was also found that across all locations, the above average yield could be associated with higher range of maximum temperature compared to that of below average yield. Principal component analysis explained 77 per cent of cumulative variance among the variables at first growth stage, whereas 70 per cent at second growth stage followed by 74 per cent and 66 per cent at subsequent growth stages. We found that coastal locations, in contrast to inland ones, could maximize the yield potential of the cultivar Swarna, due to the longer duration of days between panicle initiation to physiological maturity. We anticipate that the location-specific thresholds of weather factors will encourage rice production techniques that are climate resilient

Automatic summarization of voicemail messages using lexical and prosodic features

This article presents trainable methods for extracting principal content words from voicemail messages. The short text summaries generated are suitable for mobile messaging applications. The system uses a set of classifiers to identify the summary words with each word described by a vector of lexical and prosodic features. We use an ROC-based algorithm, Parcel, to select input features (and classifiers). We have performed a series of objective and subjective evaluations using unseen data from two different speech recognition systems as well as human transcriptions of voicemail speech

Off-line Handwritten Word Recognition Using a Mixed HMM-MRF Approach

In this paper we present a two-dimensional stochastic method for the recognition of unconstrained handwritten words in a small lexicon. The method is based on an efficient combination of hidden Markov models (HMMs) and causal Markov random fields (MRFs). It operates in a holistic manner, at the pixel level, on scaled binary word images which are assumed to be random field realizations. The state-related random fields act as smooth local estimators of specific writing strokes by merging conditional pixel probabilities along the columns of the image. The HMM component of our model provides an optimal switching mechanism between sets of mrf distributions in order to dynamically adapt to the features encountered during the left-toright image scan. Experiments performed on a French omni-scriptor, omni-bank database of handwritten legal check amounts provided by the A2iA company are described in great extent

Recognition of unconstrained handwritten words using Markov random fields and HMMs

In this paper we present a system for recognition of handwritten words on literal check amounts which advantageously combines HMMs and Markov random fields (MRFs). It operates, in a holistic manner, at pixel level on height normalised word images which are viewed as random field realizations. The HMM analyses the image along the horizontal writing direction, in a specific state observation probability being given by the column product of causal MRF like pixel conditional probabilities. Aspects concerning definition, training and recognition via this type of model and experiments performed are developed throughout the paper

Stochastic Trajectory Modeling for Recognition of Unconstrained Handwritten Words

In this paper we describe an ooe-line handwritten word recognition (hwr) system applied to the identi- øcation of literal french check amounts. It consists of three successive levels denoted as character, word and phrase level, each of them being related to the previous ones via conditional probability distributions. Training is done on character samples extracted from amount images which are modeled as trajectories in some feature space. At word level, guided by a dictionary, an internal character segmentation algorithm is used in order to maximize a global word probability measure. A stochastic grammar for a priori grammar generation probability of a phrase is proposed at the last level. Results obtained on a 1779 amounts data base provided by the SRTP 1 are encouraging, showing our system open to further improvements. 1 Introduction Because of the large variety of handwriting styles, the recognition is very diOEcult. Dioeerent categories of styles (handprinted, pure cursive) may ..